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ABSTRACT 



In this document, which points out weaknesses in 
evaluation procedures and offers a new approach to the subject, it is 
suggested that in the area of the United States studied, the Title 1 
program is drifting without direction, leadership, or systemo This 
makes evaluation impossiblOo Evaluation is sometimes a description of 
a program and often objectives are not stated in precise behavioral 
terms which would serve as a base for evaluation and which would 
dictate the programi. The document recommends the creation of a viable 
system and the establishment of active leadership to direct the 
program™ It presents four working papers which deal with the 
following aspects: an analysis of the situation as it exists, the 
proposal of a system for a state-wide Title 1 program, evaluation, 
and an explanation on why and how the writer views the system as 
realistic and possible™ [Not available in hard copy due to marginal 
legibility of original document™ ] (nl) 
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Tliis is a criticol report. I hope readers v.lll not bo offended' 

or take ny conaonts personally bocauGC ny T)urposo is not to make 

arrogant judgi:ionts from on higli, but to be constructive, Unquostion- 

. ably, thore have been good a,nd interesting 02cpcrimcnts worthy of 

• * 

support, Tlic Agency and the Ad\d.sory Council have boon conscientious 

and often imaginative in administering the program. This is all the 

more impressive in viev.'' of the inadequa.te funding and the lack of 

specific direction in tlie Higher Education Act of 1965 . 

My assigniiiont v;as to propose a nothod for evaluating the • . • 

!Eltlo I prograi'-i , ■ _ J wliavo done so , but early in my 

' . * * . 

investigation I discovered that the progi?am had a long v/ay to go 
■before any serious evaluation could bo under talc on,' In ny judf^ient 
there arc serious v/eoknessos in the present procedures, and until 
they are eliminated wo cannot o:qpoct oxiy significant progress. I 
believe we must deal v.dth these problems directly, bluntly and 
unveiled by polite conplinonts. Wo put on one face in the family 
and quite a.notlier for cho outside, .world. My comments ore Written 
as an internal report to adviso tho Agency and its Council, not as a 
public rcla.tions documont. 



My findings ojid conclusions may bo summed up briefly, 

0) At present our Title I progrom is drifting tdthout direction, 
vdthout loadorslup and vdthout system. Most of the inadequacies 
stem cither directly or indirectly from this fact. ' Among other 
things this means that v;o are not learning anything from our experi- 
ence that might contribute to hotter stato-vddo progronr.dng in the 
future, Ilnally, it makes it impossible to faco the problem of 

4 ■* 
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evaluation. (2) Tlieroforc, tho first task for the Agency is the 



creation of a viable system and tho establishment of active 
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leadership to direct the progron, Tlio active leadership must bo 

provided by the Asency^ (3) Within the context of a. systen, i't 

* ' % .. 

becomes possible to consider evaluation. 




I have tried to deal v.d.th those issues in four working papers# 
Tlio first paper contains ray analysis of the situation as it Grilsts. 
Tlie second paper proposes a system for a stato-v.ldo Title I program 

the third paper, I have discussed ovalua.tiorL# 
In a very brief final yn'oor I suggest vdiy and hov; I viov; the 



r 



system as realistic and possible, 

'ft 









GMSPvAL :JVAL''JA-rjOi]: FOP TITLE I 

Intro d uction 

Tlie ultimatG purp'-^s-- education^;.' evaluation is to nurture 
grov;th or change in individuals or groups, and the process of 
evaluation is to -aearsure grov/th cr change which occurs as a. 

f 

result of educational experiences, Ileasurenent i.’nplios the 
e:’d.stence of a set of standards and a systoui for applying the stan-- 
dards so that \ie can record relative differences I'etvfeen situations. 
We all apply many familia.r standards daily although v/e do not call 
it evaluation— thus we measure or evaluate time, temperature, 
distance, and business transactions, ‘For ’ educe. i:ion the standard is 

behavior broadly defined to, include . the emotional and cognitive as 

' * * ' ’ 

. . .. " • . ■ - , ‘ . 

well as overt action, ‘ , V • /’ 

Admittedly this” is not as manageable a standard as heat or 
humidity because hinman behavior is more complex, laore varied and ; 
more relative. ‘ Eut the principle is the same.. If i;e- v/ant more heat 

in the house we, push up the thermostat and later. evaluate the effi- 

c " . ^ hi . t‘ ’ • 

ciency of the furnace or the thermostat by checking the thermometer, • 

If pe want to change attitudes or action of peoiole in regard to 
int^erracial marriage, mo develop an educational program to achieve 
this purpose and subsequently apply, investigative procedures to 
measure the change or the program, 

I v/ill return to the question of evaluaticn procedure's in the 
third worleing paper, but for the moment it should suffice to mal^e 
three additional comments. First, in order to' measure gro\7th or 
change, the particular behaviors must be identified in the statement 
of educational objoctives — for example, it is not enough to make 
the objective; To combat white racism or even to study the question 
of interracial marriages. It must be more specific -to develop 
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sup;;port for interracial '.jiarriasesj . for vO’^zanpjo, Fm’thermore , tho 
statoiuent r.iust specify the beliavlcral outcomes expected, and these 
may range from increased av/areness or understandliig to overt action 

in sup'port of interracial marriage, 

• Second, evaluation must not bo confused vrlth the raoral issue ~ 
that is, the question as to whether interracial marriages are or- are 
not a good thing. This is a value audgmont. As such it is subject 
to examination and must bo an important consideration for higher 
education and the Advisory Committee or the Agency. But it is'not 
subject to educational- evaluation in the sense it is being described 
hero, 

Finally, evaluation must not be a onp-shot proposition conducted 
at the -end of a program, and from the Agency » s' point of view it must 
not bo restricted to discrete projects. It must be carried on system* 

‘ k 

atically throughout a program and for the Agency must be seen in the 
perspective of a state-wide program under Title I. ■ 

Probl ems 

An examination of the proposals and reports of Title I programs 
thus far nrovides no sound basis for any legitimate evaluation 
v/hatsoever. Almost without exception there are no precise statements 
of objectives. There are no specifications of beha^/iors to be 
nurtured or changed, Tho state-wide program as 'a whole operates 
\dthout system with the notable exception of the administrative 

arrangements, , j 

V/ithout question tho genesis of tho problem r-osts in the act 

itself, which sets broad s'vfeeping p-arposcs leaving specifications 
to the U. S. Office of Education and the individual states. This 
point has boo,n emphasized in the L’eport, He points out txiao 

the act has caused ‘'confusion" by combining continuing education and 






coiniiiuni uy scJrvicG suid by fcutlii.ng to dofiiiG Caig i?o3 , g of AiigiiGr 
oducr.tion in continuing education or coiii:.iunity service. It is, true 
that this lack of definibion has been a problem and has '’hGightened 
the confusion” during those first years. In the long riiUy hov/evor,; 
the vagueness is also the act’s groat strength— if i.ts terms had 
spelled out the ansuer to _ _ __ s questions, v/e would be in 

trouble. In its v.lsdom Congress decided not to say: This is 

continuing education, this is . cemi.mnity service and this is the. -role 
of higher education. More properly it implied: V/e are not sure; 
let’s find out together. In the final analysis this is v;hat the first 

few years have been all about, •' . 

In this sort of situation v/here there- aro .no rules, no science 
(no principles or bodies of knov.dodge) the approach must oo empiricalr 
From this point of departure, /iiowover, it is 'possible to proceed in 
one of two v;ays: Unsystematic empiricism or systematic eiirpiricism. 

Tlius far v:e have elected to be unsystematic. Each 

institution has gone its ov/n way conducting a series of discrete 
projects, in many cases disconnoctod one from the other even within . 
a single institution. Nothing appears to have been learned, no 
principles appear to ha-\rc emerged. Like stones rolled onto the beach 
• at high tide, they lie half buried in the sand, rarely noticed and 
unexamined for pattern or purpose that may have been imposed by 
chance. Finally , another tide picks them up and carries them back' 
to the bottom of the sea, 

A case can bo made for this random proceduro — testing, x>^ot»ing, 
experimenting with a number of approachos, a number of interpretations 
of Title I and a variot 5 " of roles for hi.gher education. But the 
process cannot continue to be unsystematic. At some point the 
em-oiricism must becorac systematic. Furthermore, if there is to bo a 
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system, it can be impos:.;d only by the K.tle I asoncv, 

ffiius far tho role of the ] Aconcy has been passive* 

There has been a sincoro effort to av/ard t>5^ants equitably. Project 
reports have been dutifully collected and filed. There has been 
only minimal fulfillment of tho federal requirements for a state 
plan--by merely designating the emphasis on local government the 
Agency underscores its intention not to j.ntorveno a.ctively in the 
stato-i.dde program. The only point at which the Agency is not passive 
is in the supei-'/ision of the administrative rod tape imposed by the 

Office of Education. It goes vdthout saying that this is important— 

/ 

and it is my impression that it has beon done eirbremcly woll at 

' ' I e f 

-but it does not change tho fact that tho Agency *s role has been 

essentially passive o ^ ‘ 

% ' 

There appears to be ,a .strong feeling, both on the staff and the 

, ^ - » 

Council that the condition is explained by the lack of staff — tho 
need for a full time person. This is unquestionably a pressing need, 

but V'o are fooling ourselves if v;e accent this half-truth as the sole 

• I 

or even major explanation. In my judgiTient, even a part time person 
could operate difforcntly if ho wore to start from tho base of a 
clearly Gstablishod system and tho assumption of an active role. 

It is worth noting that most of the program directors inter- 
viewed by mo also viev/od the Agency as playing a pa.ssivo . role. 

Tttle I is seen as another pork barrel, albeit a very small one, 
v/horo institutions of highor education may dip in for their share of 
the funds to undertake discrete projects, Tliey accepted emphasis 
on local govornmont as nocossary to comply vdth terms of the act. 
Dopending upon thoir institutional interests, some v'ould like to soo 
tho emphasis on local govornmont continue and others would like to 
see it changed. Ih responding to my question regarding tho proper 
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rold for tho Agoncy, it was cloar that projoct directors looked for 
little more than monoy ^ and guidanco in dealing v/ith the red tape 
inroosGd by Washington.’ One director, conmonting on tho role, of the 

Agency, volunteered to rani?: directors , rejected one of 

them out of hand because he v/as ''too theoretical. 

In spite of this general satisfaction v.lth a passive, un theore- 
tical agency, I believe it misses a major purpose of the legislation. 
I suspect the vi..sion of the lavnnakors v;as much closer to cooperative 
extension than to a pork barrel. This moans vro rdust have sound 
■principles, including a cloar specification of state-v/ido objectives, 
and a system in which the role of the agency is', active rather than 
passive. Until this is done there is no't'much point in talld.ng about 
evaluation or about improving the quality : of 'the sol action process 
and effectiveness o’f Title. I programs 
Evaluation, of the Present Program 



In view of tho problems as I s.oo them, what can be said about 
the current programs? In the absence of clearly stated objoctivos 
prior ovaluators, both formal and informal, have tended to api:>ly 
their ovrn criteria, sometimes stated, more often unstated, and 

frequently imposed after the fact. 

•rxie , Report is a case in point. After emphasizing the need 
for more precise objoctivos, tho report starts off bravely vdth a 
couple of randomly selected interpretations of Title I. But in tho 
end it resorts to its own yardstick— ” the structures program) 

developed for meeting its ’ objoctivos. » It is important to recognize 
that this is not an objective of cither the Agency or individual 
■progr.iwms. It is the aquation: If the structure is effective, 

the program is successful. If it wore an a.cccptaolc standard, it 
could still servo as a basis for future evaluation. In my judgment, 



however, it is not adequate. At least as it stands alone it is 
highly mechanistic, grossly oversimplified and potentially dangerous.. 
One simply cannot evaluate a program on the basis of e:cclusive 
concern for' structure or process vjithout any regard for substance, 

It maizes it possible, for o:tample, to compare prunes and oranges and 
to decide in favor of oranges because its easier to eectract the juice 
oven if the special quality of prunes is uhat the doctor ordered. 



***«■* 



i3val nations by the Title I directors \-orc the most disappointing 
aspect of the 'Title I program. My discussions v.lth agency daroctors 
from other states and v/ith the staff in Washington confirm 

• ' judgiaont : ' Our program as a whole leaves much to bo 

desired, but it is no better and no worse than counterparts in other 
states. If this' is damnation v.dth faint praise, hit need not- bo. 

As already suggested the early years v;cro bound to be devoted to 
exploration, to encouraging experimentation in many directions. 

This is justifiable if v/o learn something in a systematic v/ay thatf 
vdll help us achiovG a sense of purpose and diroctioii for Title' I. . 

Tliis is what I looked for in the project reports — and by and largo 

/ 

it simply was not there, Tliis docs make the faint praise damning — 
and it does not help much to knov/ that other states are in the 



same boat. . " , - , • . 

Tno root of the problem rests in the inadequate statement of 
objectives. An anaJ.j’-sis of some forty progress that have received 
Title I support reveal that statements of’ purpose fall into the 
follovdng categories. (1) Tlioy a rc so broad and general that it is 
imposs i ble to iden tify _ any mc as uroablo outcome s. For cxomple, one 
prograTii receiving a grant stated its purpose w-as to aj,ort the commu- 
nity regarding the problem of a grovdng aging population and to. 
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educate social v/orkers, wblfaro porsonnol city officials and civic 
minded citizens rosarding the latest findings and programs in the 
field of gerontology, 

( 2 ) . Objectives a r o stated in such c osni i c , ambitious terms t hat 

they are completely out of touc h vdth rea lity For example, the 
purpose of one program v;as to solve the problems of poor community 
leadership in somi-rural areas. To such a grand purpose one can 
only say, ”V/ov;!” . • 

( 3 ) Objectives are stated i n t orais of tho program , 

One purpose, for example, vras to provide a scries of v/orkshops' for 
directors of volunteer sorvicos in all settings that. use volunteers, 
providing them v.dth consistent, systematic training to improve their 
creativity and'^offectivonGss and thereby improving tho volunteer 
sc3.'*viccs \ 7 hich they suporyiso. Among other .things, this is too 
goncral to bo meaningful,’ but more significant, all it says in 
effect is; Our purpose is to run a program, 

• ^ ^ 2]^ jectivQS are phr asc_d as a r es ta t ome n t of tho ^nood , 

In ono case, a proposal claimed there v/as a need for community 
develbument to nrovido coordination and leadership, Hion it v;ent on 
to say that tho purpose was; To bring together leaders of the 
community to' coordj.ua to efforts and to offer 'nev; pro ^^ams that v.dll 
develop community loadorship and citizen. imxDrovomont._ 

Attempting to evaluate a x>rogram oh the , basis of inli blots such 
as these is liko using a garter snako as a measuring rod. Svon vdth 
tho best intentions in tho world it is not surprising that project 
directors have failed to produce very useful or ;;ioaningful reports. 
In many cases it is impossible to avoid tho clear imiprossion that 
they cither do not understand the evaluation jprocoss, or choose 
to ignore it, viov.dng tho ro]port as a formality required in order 
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to receive *thc final pa;;>'inont on. the grant 






In either event genuine evaluation is inevitably alraost’ 
non-e^dstent and meaningful reports are fov: and far betoeen. 

.Generally, reports may be classified as follov/s, 

) i^'ie Iio..adcount , Tlie numbers game is a favorite preoccupation 
of report v.Titers. Tliorc arc onthusiastic accounts of participants 
reckoned up by dozens and v.dld inferences of others reached indirectly 
Those who simply cannot claim herdcs of students :.iay talk about the 
small, selective audienco. In part the headcounting syndrome is a 

. - ' '4 

reflection of the marketplace mentality often found ariong adult 
educators, but it is . encouraged hoiro by the emphasis on cost per 
■participant in the "J, S, Office of Education. • Tlio nuiiibor of people 
reached by a progr.aiu is not unimportant, but educationally speaid.ng 
it bocomes • relevant only in relation to oduc.ational objoctivos. 

Out of context numbers arc at best meaningless and at worst misleading 



distractions from the substantive aspects of the oducationc^ oxpor- 
ienco. 






2) The harpl neG S cou nt. Another favorite question of adult 
qducc.tors is; Is everybody happy? Did you like the professor? 

Was the meeting room comfortable? \7ould you prefer to have more 
'clWs'“s^G^ibns' Do^ you Tb'31'' satisfied v.lth the program? 

Do you'tiTink it v.lll be helpful? Like hoadcounting, this is part of 
the markotplaco mentality which .puts selling education on a par 
vd.th soiling a now container fo.r iiost toasties* Again it is not that 
such feelings or opinions are unimportant, particularly vdth adidts 
who arc not captives of the formal school system. Dut they become 
significant only in the context of many factors in the total evalua~ 
tion. In some cases the evaluation reports consisted of a happiness 
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count .?.nd nothing more. 

t 

3) TliO circular ur^ur.iont . This form of report and evaluation 
is urob.ably the r.iost popular* It goes so.icthing like this: The 

cov.rauiiity has a particulrT need; vc propose a prograa to servo the 
need; v/c ran the lorogran; tlioroforo, the need has been served. 

Neither a need nor a. prograo is synonymous ’.'ith purpose. Further*- 
r-oro, it is inijossiblo to csca|)e the conclusion tlio.t on ..iO.ny occa.si'.-'Bt- 
there v/as little or no coimoction botv/ooii tno need a.iid the progra.i*i. 

In effect the institution jacked up a nood and put a pro gran under 
it, a proGran that they v/anted to run anyvfay. In any event, this, 
sort of reporting tolls us nothing and perhaps is- uiorc useless than 

head er happiness counting, •. -i ' '/ 

Aj.) D escriptive reports . In' a number ;of bases the ovaluataon 

takes the fern of a description of the prograxU, Although thorc is 
groo.t v.?j*iety in.tho quality and usofialnoss of tnosc reports, the 
aorc thoughtful statements 'do provide insights that could help shape 
future prograriiaing, • Perhaps .the :ios;t effective reports 00:0 those 
that anguish over failures or ina.dcqua.ciGS, Sene of the explanations 
for failures arc incredibly naive, 'For example, it really is not 
necessary to invest ^|:10,000 to ^512,000 'bo learn that adults are 

rietiva.t :.d a'hen the ria’.torial is inincdiately rolovaait, or that it 

*■ * ' , • 

. is difficult to Jiold adiits ta the convoni^iona.l-acadonic schedule 

I ■ 

of one class pen' week for fourteen weeks, Novcrtholcss , there is 

y- 

neat in many rejports. If the data v;cro analysed, it could bo used 



to ir-pro VO tli e 



quality of stato-vddo progranning. 



5) Tlie t?\igiblo product , In a^ few ca.scs the outcome and in 
effect the rone *t has tal^cn the fern of Ow tangible product, Tlius, 



ColLgo can shov; us a rofcronco library, 



Coll ego 



a colloc bive barV:JLning najiual; 



College and thc_ University of 
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have produced a sorios of manuals doaling v/ith charter 

» 

revision. Those arc solid accomplishments, but educationally 

r 

3peald.iTS thoir eras tone o does not demonstrate that any chango or 

gronth occurred. It may bo, argued that such projects represent 

community service and that this is sufficient. If this is to be a 

policy of the Agency, it should be clearly stated and objectives for 

the project should be specified. In other v/ords, it is not enough 

to state that the purpose is to prepare a manual or set up a. library. 

The Agency should knon v/lia.t purposes the project is designed to servo 

TIio rand o m plus . Another kind of report has boon the- .report 

of beneficial outcomes which', if not .entirely unanticipated, cxcoodod 

reasonable expectations. In some cases these extras have .v.c-aing 

» * 

• y t ' 

to do v/ith the purposes and are of dubious va.lue as community' .orvicc 
and continuing education. ,It is nice to 'knov;, for example, ' that a. 
progrcj.1 was visited by foreigners and applauded by a national asso- 
ciation, But it does not show ho\! or if the community benofitod. 
Particularly v.'h on tho answer' is not provided olsowhore in ’ the report, 
vio arc bound to ask: Is this a justifiable 'oxpcndj.turc of :;:1 5,000? 

Similarly thoro aro questions about other random benefits’ such as 
favorable publicity complete v.dth newspaper clippings. 

In other cases tho extras represent impressive gains for 
continuing education and community sorvico and appear to bo in lino 
v.lth 03 cprosGcd or implied objectives of tho Agency and tho Advisory 
Council. Tliero is cvidcnco (cither in the reports of from my 
intor-'/iews) that Title I projects strongthoned institutional commit- 
ment to continuing education and community sorvice. At __ 

. iUtlo I projects have quickonod and strengthened plans to sot up a 

continuing education division. At College, the Title I 

grant was a crucial factor in enabling tho creation of its Municipal 



n 



Rcforonco .3urGi?.u, 

In another diroefcion 'Utlo I Tprojoctn hayo onablod inctitutions 
to commonce programs that will extond- beyond tho ^.^rant program, 

Tlio corx.iunity otudy at Colloso has boon 

follov/od by a second study a.nd by planning of courses to inplcaont 
tho findings, A litlo I grant to Univoroity has helped 

launch a graduate iDrogron that probably v.dll continue long after 
tho grant has boon spent. In , tho • __ _ .r Center 

for Coniaiunity Studios appears to bo v;oll on its uay toward bocoming 
a pernanont institution, ' 

Another extra in the ’ '..•r)xp6rii.iont has boon 

offoctivo intor-institutional cooperation, Tlio Ad-'/isory Council 

has looked with groat favor on joint xDrojocts and has booh the 

\ * 

catalyst to bring institutions togothor for comoion undertakings. 

It may bo -tho major exception to tho passive rolo and shows vdiat can 
bo accomxDlishod by an active agency. 

Finally, some litlo I projects a'g’joo.r to make a positive impact 
on tho community, i"ho ovidonco is rather thin, biit at least it is . 
possible to soo liopoful . signs. For example, • claims 

^that the Titlo I project made tho community more aware of tho collogo 
as an institution concornod with community sorvico o.nd community 
education. In another case, tho ' * project appears to have 

inspired tho creation of an on-going citizens' committee- concornod 
vdth regional planning, 

Tho Evaluation Procoss and Progr am_ Dovolop mont 

Statemont of objoctivos , Bonja:.iin Bloom, an evaluation 
authority from tho University of Chicago once commontod that a 
sto-tement of objoctivos in prociso, behavioral terms can s'orvo 
two purposes: It is a base for evaluation; but more immodiatoly, 
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it dictate's prograa. In other v/ords, if you aro prociso about the 
outcomes you v/ant to acliiovo, It bocorioo easy, or rolativoly so to 
design educational cxporioncos that ’./ill a.chiovo youjr objoctivos# 

Tills obviously is not possible vdth o.l:'.iost all our Title I 
prograns. In fact, as I have pointed out, it often appeared that 
proposals v;cre croatod by starting in the niddlo v/ith a pet xirograu 
or favorite method. Tho fine sounding statements of needs and . 
objectives appear as an o.f tax' thought to saeisfy roquiromoncs. Even 
if this is not the v/ay it was, tho fact romains that none of the 
proposals state objectivos in such a way that they give directions 
regarding substonco or mothod of the • educational cxporionco. It is 
nothOGsiblG for tho progranmor or ■ the teachor- to say: If these arc 

’ I the kinds of people to bo roachod, 'if tlioso aro tho outconos you 
v/aht to achieve, th'on those aro the educational cxporioncos I \.all 

introduce and these aro the mctnocis I "ill use. 

One possibility, of course, is that tho program director or thoy 
toacher had objectives in mind— they merely woro not stated. I 
tested this hunch in some of my intorvjlows , but found little 
cvidoncc that tho concern for objectives or outcomes nad gone deeper 
‘-than the suporficial statemonts incorTDorated in tho iiroposals. In 
one ease I askod tho question directly: In tho back of your mind 

t 

do you hv?.vc specific outcomes you aro trying to acniovc? 

The ansv.'si/r \?as an immediate and categorical *’No,** 

Under lyi n g A s sui^ tipn^, 

CloGoly associated with the statement of objectives there ^ 
is tho question of liypotlicscs or assui.iptions underlying tno project, 
I aiu sure thoro aro assumptions bohind all Title I projects, but 
they are either unconscious or superficial. Unless tho nypotnosis 
I is clearly stated and consciously incorporated in the program, y;c 
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cannot tost it and cannot loam anything about its validity that 
could bo applied to future prograi’.is, _ 

Superficial hypothoacs are roa.lly not v/ortli .testing. It is not 
enough, for example, to assume that fina.nco committees need to update 
knowledge in order to handle municipal budgets — at best this is a 
clichi rooted in the fallacious notion that more specialised knov;- 
lodge’ is the solution to all our problems. It is not my purpose to 
question tho assumption, but I do aondor about tho validity'' of - a 

I 

prograiu based on 'this cliche r.eceiving federal support. 

Tills is not to imply that . should not h.avo recoivod' 

support from ^Utlo I, but I boliovo one purpose' of tho program should 
be directed toward testing -some hunches or hypotheses. It simply 
is not enough to say v/c used tried, conventional methods, based on 
conventional hypotheses and are pleased to report v/o have thirty or 

forty contented finance committee members in ’ ' ' _ 

Knov/lodgo about comm unit y pro b lems . 

A third difficulty appears to bo lack- of knov/lcdge about 

$ 

tho problem that is being at "backed and .about tho co mi:! unity, or 
potential adult student body. In one case tho program proposal 
frankly stated that the first phase would be research and investi- 
gation of the literature. This was to be used to build a library 
and to dotormiiio how'a planning region sliould bo defined. . l^thout 
such basic knoifledgo it is not surprising that an institution has no 
hypotheses or objoc-bives. Indeed the introduction of plans for an 
oduca.tionsJL program bocomos somov/hat naivo. 

Other institutions v/oro less frank, but in p>3?ogr.am after 
program it is apparent that tlio analysis of the situation and 
hypothetical formulations v'ero entirely inadequate. In one case, 
a program v;as so out of touch vlth the situation that it did not 
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1 

• ©veil rocich tlio urbo.n police force v/liich v'cis the original tcirge’c and 
for which the noiuy had been given. In many other ca-eos, the 
mid-point ''progress reports*' rovoalod that instead of being half-v/ay 
done, "orograus v/ere just starting because there v/as so much learning 
and planning to bo done, 

Tliis sort of activity is perfectly legitimate, particularly 

in an area like community service v/horo we have so much to learn, 

I believe it is unfortunate that the Higher Education Act of ^ 

1965— at least .'as interpreted by the, U. , 3 . Office of Education— talces 

such a dim view of grants for study and roscarch which is so' neces- ^ 

- • • 

' ' y 

sary for effective program planning.' Given the situation, howeverv, 
it becomes even more important that v/hatever may be learned from 
each and every Tibi o I program must bo integrated into a. stato-v/ido 
body of knowledge or sot of principles regarding community service 
and continuing education. One of the unhappy facts of Title I 
programs thus far is tha.t each project starts from scratch to redis- 
cover v/hat could be part of ' a cbmimon body of knov;ledge. 

The Agency 

One of the impressive aspects of Title I . is 

the sincerity, the sense of responsibility and the energy of botli... 
the staff and at least those members of the advisory Council who 

‘ ■ i 

■ have boon active. In spite of this comi\]itmcnt, however, there;, are 
problems— and no one recognizes this i)cttor' than the staff and • 
Advisory Council. At Council meetings and ill personal conversations, 
staff and Council members openly acknowledge inadequacies in the 
selection and evaluation' of programs. In large measure the Agency's 
problems arc inherent in the inadequacies v.iiich have been discussed 
already. In part the problem rests v.lthin the Agency itself where 
there is the somio fuzziness and lack of clarity regarding statc-v/ido 
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objoctivG,^, hypothc^:oG, and knov/ledgo of tho field. 

In thinking about^ statG-v.dde objectives, I believe a number of 
issues must bo considered. One has already been raised: Shall the 

Agency »s role be active or passive? As I have already indicated, 
in my judjpiient tho Agency must bo prepared to assuiee an active role. 

In tho remainder of this section I would like to discuss four addi- 
tional that I believe are barriers to an effective state-v/ide 

program, 

1 ) Thus far no v/ay lias boon found to,, use any of the data or 

anything learned from the Ti,tle.I, projects.’ According to 

" •* * • 

he- and I arc tho. oni^': two persons who have 

oven read the complete files of the programs lor the fisca.l years, 
1966-1968, Even the xoroposals have not bGcn-'rea.d completely by more 
than a handful of ' the Council' members who ha.vo served on tho 
Selection Committee, 

It goes v/ithout saying that by and large project directors are 
unaware of Title I prograias' of colleagues in -tlio field, -They do not 
even knov/ what the projects are, not to mention anything substantive 
about thorn. To carry this to its extreme, the Director of the 
Bureau of Public Affairs at Col?_ege v.ras not even a\/aro of 

other 'fi-tlo I iprojocts at his own' ins'titution. In another case, at 
Cnivorsity the results of one project v.'ere not- used or ovon 
considcrod in any v:-ay in d second projcct^^claimod it \vas building 
upon tho results of tlio first experiment. 

As I have pointed out, in spite of the shortcomings of tho 

\ 

project roxDorts, tlioro is much to bo Icarnod fro'i:i some of thorn and 
probably something from aj. 1 , Tho organization and analysis of this 
material could contribute to a body of kno;;ledgc about community 
ser-'/icG and continuing oducation. It could help tnc Advisory Council 
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in molding judginonts about individual projects and sotting state-v/ido 

objoctivos, Tlic collcctivG c^cporionco could liolp individual insti- 

' > 

tutions in futuro progrcuaiuing. It could obviat’o unnocossary 
duplication. 

To toko ono oxani^le, the _ College under tookr 

a survey of economic ncods and op.portunitios as a prelude 

to development of educationaJ- programs to servo these needs. To 
some oxtont if the findings v/oro generalized, other institutions 
serving similar sized cities should bo able to devolop progrpmis 
v/ithout mald.ng a similar survey. At a minimum the procedures and 

r . ♦ If 

. / 

findings should bo available for co'mparativo' purposes, Tlio report 
lies unused anyv.iioro exc'cpt in 

In another case, I had occasion to 'talk v.dth ' _ about 

jphartor Revision Pro ject. being offered by - _ . College and the 
University ’ ' •, , At the time was not sure hov/ 



the tvra institutions should organize the tolovision discussions v/hich 
are part of the projoct. Because I had recently talked to 
about his T.V. AssUisbly, I v;as able to suggest that 

College might bo able to use "listening posts,” v/as 

not really familiar v.dth the T.V. Assembly and the idea had not 
occurod to him, I do not know. \;hothor anything has, vdll or even 
should materialize, Tliis point is that> this kind of. bridge must 
esdst and be exploited systematically. 

2) In this same connection, ultimately the Agency must be 
familiar vdth or have means of finding out about major developments 
in community service and continuing education outside of the Title I 
program, Tliis is more complicated and more difficult, but it can 
be achieved gradually. It is obvious, I believe, that knowledge of 
other continuing education progr-ams can contribute to Title I -projects 
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and can lielp avoid duplication. 

Afjolin ’bo cito oiio^ cxcU'-iplc, ,■ lias roccivod tuo 

grants to support seminars for mombers of tov/n financo committees. 

boliovos that these are much needed, original and successful. 
It is also claimed that the tuition must bo undorv/ritton, Tlic 
Advisory Council has generally supported this position and has boon 
impressed by the program. In a very cursory roviovf.of the total 



program 
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a course on budget and finance for municipal officers. It has boon 
offered for sevoral years, and the tuition charge is fifty dollars. 

.f 

' ■ • , * ' * . ^ ’ 

According to . ^ the course has -bobn talicn by tov/n finance 

coneiittec members. I did not pursue thiis _-bcyond these superficial 
facts. A more 'thorough examination might ’ demonstrate that 
is mald.ng a distinct contribution. Perhaps it is necessary to charge 
no tuition. Without information about the ■ ' ' e program, 

hov/over, the Agcilcy is unable to mal^c' compa.rative judgiiionts .or con- 
structive suggestions to strengthon the experiment. 

# 

3) Tlio Agency has been unv/illing or unable to come to grips 
v.lth the rolationsliip of. higher education to terms that a.re crucial 
to lb. tie I, h^iat is tliG rola.tionship betv/con, higher educe. ti on and 
community service? h^liat is the diffcrcnco bctv/ccn consultation and 
oducation? Are both logitinia.tc concerns “of higher educa.tion? Wliat 
has '-problem solving” to do v.dth higher education — and indeed, v;hat 
is a problem for tlio purposes of Title I? 

Cortoinly from tho point of viov/ of higher oducation the defini- 
tion of a problem must go further than the laundry list of specific 
needs which can be compiled from -the proposals to date. To remind 
you of a commonplace, jDrpbloms arc not all of the s.amo order — they 
may be very sjpccific and immediate or very b.road and encompassing. 




To uso tho CC180 just citod, bud^oting practioo is part of a larger 

I 

problon of municipal ^finance uhich in turn is part of tho crisis of 
tho city as a social organization. Similarly the immodiato problems 
of housing or political participation in tlio ghetto arc part of 
larger problems of urban living. 

As I have already pointed out, the tendency in tho Title I 
progran has been to deal in discrete projects. Ono of, the major 
outcomes of this approach is that v;c never see problems v/holo. 
Obviously for oducationo .1 xourposes v;c must broak donn tho v/hplo 
..problem into nanagGablo proportions, but they ilitist bo viewed 'ah d 
developed as contributions to a larger whole;' Otkerv/ise wo arc 
pouring our money and efforts into a bottomless pit. It is the 
responsibility of higher education in general and tho State Agency 

in particular’ to' provide this perspective, . 

I found it disappointing that tho university consortium under- 
written by Title I appears to bo taking this same fractured view of 
problems in parts rathor than wholes, Wlion I ■ intorviewod some of 
tho participants, I found nono of them had ovon considered the notion 
that problems night be of different order, and tho project v/as moving 
ahead v.dtli tho discrete, oasy-to-got-at, nuts-and-bolts issues. 

the consortium mot with a number of public officials 

to ask them for holp in identifying major problonis. ^ ' 

began tho neoting 'b.y suggesting a, whole jproblom— he talked 
about survival of tho citj^ and about tho neod to c^camino municipal 
functions to deternino what could be performed by tho contemporary 
city and what should bo performed, by some other agency. As cjaamiDlos 
noted a number of particular issues that wore related to 
this larger problem, V/lion he had finished, members of tho consortium 
and other municipal administrators proceeded to ignore completely the 



largo x^roblom, Tlioy latched on to his particular Giianiplos , 
added a fov; more of ;thoir ovrn and discrissod thoni all without any 
roforonce to the larger context. Only one riiev.ibcr of the consortium, 

appeared to want to follow load; when he 

Y/as shouted down, lie joined the game of listing particulars. 

In the formal report of the mooting, riiolo problem 

was not even mentioned. ^Plie list of thirteen problems emerging from 

* » ’ 

the discussion is roxirodiicod in full. . ' 

- 1 ) Municipal Finance 

’ 2 ) Improving rogioiicril arrangements • - 
■ 3 ) Short tenure of mayors (2 years) . , ' 

k) Public education eosts 

3) Shortage of troi.ncd personnel ' for, municipal governments 

6) Lack of standards .for measuring the performance of depart- 
ments ■ • • • ' - ''v J ‘ * • ■ 

7) Need to incorporate standards bf evaluation into the 
management reporting systems • 

suggested use of comiDut.ors to 
design useful systems 

8) Sources of additional funding for- cities 

9) • Understanding Mayor ®s need for and preparing budgets 

10) Lack of Public's understanding about what they are getting 
tl ) Municipal needs for standards, structures and staff 

12 ) Mayor’s lack of time to run the city 

- problem of sorting out priorities 

13 ) , Logi.slature passos .acts v/hich ovor-burdon the cities 

financially ■ 

I have grave doubts that some of these problems lend themselves 
to an educational solutiori. But assuming they do, hov/ do you. sot 
priorities v.lth a list like this— talcc a popularity vote? Ono of the 
disarming aspects of this approach is that it can givo us a false 
sense of accomplishment. It is like a bunch of Boy Scouts rushing 
through tho forest randomly chopping down some dead branches they 
happen' to see and can roach. At tho end of tho day they aro tired 
and happy over their good works, .but the forester kno’ws they have 
done essentially nothing to prosorvo tho forest. 

Tills short view is not limitod to tho consortium, but applies 
to the total program. It is not surprising that ackiinistrators. 
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•harried by the pressure of day- -to-day tasks often tal^e a short view 
of their problens. But educators should know bettor and certainlj'’ 
the Agency uust teJ^e a responsibility^ for seeing it whole, 

4) There luust be a concerted effort to locate other agencies 
or organizations that r.iiglit cooperate with Title I on matters of 
common interest. Cooperation might take the form of joint projects 
or supplementary financial support. Tlie latter is much needed in 
viov/ of the limited amount of money available and the restrictions 
rega.rding its use. Although this is not a. simple task, I do not 
belicvG it is impossible. 

t ' ,4, ' 1 ' • .,/* 

To t.ake one example,' the lav;, and its inrtorpretation by the 
U, S. Office of Education make it extremely difficult to undertake 
rcspa.rch under .-.Title I, Yet in ma.ny.aroa.s research is dcsp^ately 
needed. In the course of my invostiga-bion I' talked v/itli 

■ ■ ’ . lie saw no 

reason why some institution could not receive support from his 
program for educational roscarch ’ that y.'as rolated to community 
scrvico and continuing educe. -bi on. Similarly, wbon I “^/isitod 

, I found intorost in 

exploring possibilities for cooperation. Both of those arc a long 
way from firm commitmonbs, but thoy do indicate intorcsting 



possibilitios. ■ * -' 

All of this may scorn overwhelming in thd faco of limited staff 
and limited funds available to the Agency. Certainly this is an 
important reason why littlphas boon dono. On the other hand, a 
beginning can be made v.lthout too much offort. Tho discussions 
nentioned above did not tak:c very long— lunch 

and about thirty minutos This is not a reflection on the 



staff 



but a return to -an earlier point. So f.ar 'Title -I 
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has failed to develop a clear Gc-nsc of direction 
and a syston to ir:ipl9r4cnt the progran, 

Sui-iiMary 

In this uorldLiig paper I have atteaptod to indicate tho direction 
in v/hicii I iDoliovc the progrcai i;iust move. It involves; Seeing the 
probloms of community sorvico and continuing education v/hole; sotting 
objectives in precise, mcasur cable t-rr.is; dovclo-pmont of a sound 
thooi'otical base; and the construction and dis'semiha tion of a body of 
knov/lodgo. This can bo accomplished if tho Agency assumes an active 
role to provide leadership, coerdination and integration of! a, 
stato-vrido effort. 



A SYSmi FOR THE - ' ■ ^ PROGRAM UNDER 

TITLE I OFrTHE HIGHER EDUCATION ACT 



Introduction 

In this second v/orldiig puper I want to discuss the elements 
of a system to implement the progr'oia in the directions sugc^^sted in 
my ev.al nation. As I have already implied, in ny judgment’ the 'crea- 
tion of a system for the operation of Title I is crucial to the 
dovolopmont of a more effective program of community service and 
continuing education. In many nays it is more crucial thah the 

evaluation process, because it edmos^ first and. is prerequisite to 

' < ' • 1 

effective evaluation and prograia development, ^ 



System may. bo an ill-advised term to; e^cplain • v/liat I have in 
mind. It has many emoanings and can bo very complicated and highly 

"■"X 

technical. Still, it is a good word to emphasize what needs to bo 
done to provide an organized, logical \roy to deal vdth the various 

factors related to the Title I j)3rogram,. Actually, what I have in 

*' ' * . 

mind contains the corraon olomont found' in all systons— tho integra- 
tion of diverse factors by moans of a unified methodology. From a. 
technical point of view the system for Title I loans tov/ard cybor- 
notics in that its concorn must bo for processing information 
rather than transforming energy. 

One of tho objections to a system for Titlo I ■ • • 

may bo rooted in tho conventional v.isdon: ' Neither staff nor funds 
to implomont tho systom arc avrilablo now or in the forosocablo 
future; theroforo, it is not practical. But I soo the system first 
as a model or goal tov.ard which wo are striving. With tliis in 
mind, I soo tho systom as sufficiently flo:dl,blc to permit us to do 
vdiatcvor is possible vdth whatever rcsourcos are available, Vfnatever 
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•70 do, hov;ovar, ,nust fib and help build tov/ard the long range goal. 
It 30'x.iG to me that thi? v.lll accomiDliGh a numbor ox things » 

1 ) It v;ill give a sonso of direction, v/hich is nov; lacking'; 
and it vdll otrongthen our ability to establish policy and provide 
consistont leadership to higher education. 

Z) It v/ill provide us with a framov/ork for nailing our empiri- 
cism’ systematic and establish a base for con'binuing .evaluation, 

3) It vm.ll provide us v.ith an orderly v/ay to store whatever 
kiiov/iodge v/e collect so tli'.vt it c.aii bo applied, 

k). The o:d.stonco of a system and an indication that the’ • 
Agency knows where it wants to go rill b'o tho'most persuasive 
argument wo have to convince legislators' pr others that- we should 
receive more financial suiDioort, It is oho ’thing to v/ring our hcands 
and \!?±lt Thorb is so Viuch to do if wo only had more staff and 
moro money. It is quite ano’bher to be able to shov; precisely what 
is noodod and’ why, • ; 

■ 5 ) linolly, since the alternative to a system is the chaos of 
no system at all, v.iiat can WvV- lose? 

How the .3yste::i Works 

To help you visualize the system as a wholo I have prepared a 
chart v;hich appears on page Zb,,. As the chart indicates, the system 



is visualized as performing throe functions, hold together and 

. •» ♦ ♦ ' ' *1 ' 
directed by the agency viiich'is placed in the middle, . 

1 ) Tlie first step in the process is the’ development of ideas, 

‘ ** . " 

hunches, hj’-pothosos, Tlicy may emerge from institutions of liighcr 
education, the .agency, the community, or some combination of the 
throe, 

2) Tlic no::t stop is the technology, Tlio idea is translated 
into a plan for an cctucationol program— this is the proposal which 
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prosonts tho hypothosoc, tlij objoctivoc or ■ oz.-poctod outconos, and 
and tho substanco and oothod of tho educational cxoorionco. Tliis is 

f * 

probably done by tho institution of higher education, but tho 
conuunity and tho agoncy may participate, 

5 ) Finally there is the field exporinont carried out by the 
college or university in a particular conaunity. Again the program 
is operated primarily by tho educators, but the community and the 
agency may particiioate, 

L\) In tho center v/c have the agoncy. rxiis is tho feedback 
for our system. It processes tho information, but in our system 
it must ’do more than that. 'You r;ill 'rocoll that tho word, cy borne- 

’ , ' . u,*V . . ’ * . 

tics, comes from a Greek word moaning otoorsnan. Our steersman is 
no automatic -pilot with pro-doterminod settings. 'Tho agency is 
Cciptain and navigator, constantly evaluating tho course and 

iU„. ^ 

03Corcising human judgment as it intorycncs at all points in tho 
process to lorovido leadership, ddrection and support. In other 
words, it is an active agency, not a passive one, 

5) ‘5^10 system is enclosed in a larger circle which is colled 
tho cormnunity, Tlio community is not part of ’ tho oducationed system 

indicated by the arrows, it is involved at -all points. 
Incidentally, this formulation resolves the questions Wliat is tho 
community? It is the organization or group of people we happen to 

4T ^ , 

bo serving in any given situation. As such it is ubiqidtous, always 
involved. At the saiuo time it never has the specific structure 
required to assign it a function or responsibility as an integral 
part of the system. 

6) Finally, tho system shifts tho emphasis in community service 
and contmuiiig education xron aosolutos to process. So far, nrogroiiis 
developed under Utlc I— and indeed, in most continuing education— 
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OXQ based dn absolutes v/hich take tho fora of a linear iDrogrossion 
which soos sonc. thing like this. V/o have a goal to chango or inprovo 
sor.io aspect of co;:i:’unJ.ty lifoj fexets arc gathered regarding heeds 
related to this goal;' a progran is planned and inplonontcd. Any 
• connection botv'oen this approa.ch and iaproving the quality of 
cooinunity life is coincidontol . It is purely acadoaic and nay work 
insido tho fornal educational systen, but is not relevant for 
connunity service and continuihg education. 

In this systen tho emphasis is on continuing education as a 
process. It begins vdth hypothesizing— the creation of ideas that 
arc tested all tlib v/ay round the circle throjogli' the technology and 

r •*. / 

field inplcncntation. ‘ Tlio ddoas may be rejected and undoubtedly 
will be' nodifiod at any point as they cone' up' against the roa.litics 
of teaching and 'connunity life, Ultir.ia.toly an idea nay come full 
circle to bo rostructurod and sont around again and agaaii— and 
perhaps again. 

Until wo can crc.ato a systcu with a food' back nochanisiu, little 
progress will bo made toward building a process that mai^os a viable . 
connection between higher, education and the quality of community 
life. 

-a 

In Roga.rd to Flo!-ri . bility ' . 

, At first glance tho system may appear to be highly inflc:d.blo. 
Actually I do not think it need bo boca.uso tho process is circular 
not linoar , A linca.r system \7ould presumo a sot of fixed stages, 

in this case running from hypothesis through technology to field 

• ■) 

experiment. In a circular system wo can start anywhere. Ideas 
may emerge from random experiments vdth tochnologj'' or programs. 
During tho early stages this is unquestionably where tho process 
vlll begin because that is where we are, Tho point 'is that xlth a 
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syston v/c nood not worry that current projects oto randori bocauso 
the data and export once can be applied systoiiiatically, 

Sunnary 

I believe this syston need not bo created out of v;holo cloth 
but can bo developed gradually. Perhaps even more important, it 
should bo .possible to enter the syston at nany points, follvoi/ing 
the path of least resistance, if you v.lll. Gradually wo could 
expect to build a body of knov/lcdge about connunity service and' 
continuing education and to see problcns of the ConnomvocJL tli 
whole — and from this beginning we should bo able to novc toward’ 
norc precise objectives, a sound theoretical "base and an incroasin^y 

^ « f p 

t * ■ ^ t ■ 

effective state-vdde x)rogram, • 
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III 

SVILUATION. 

I 

In ulio third working paper I want to discuss the evaluation 
procoss. As I have pointed out, evaluation does not noJee much 
sonso unless wo know wlicro v;o arc hoading with K.tlo I and unless 
wo have a systoa to inploaent our purposes,. Halving considorod 
directions and a systen, I believe v/o arc ready to look at evaluation 
SvalUo.tion is inevitable. Viliorc chjico is involved or v/horc 
relatj-vo judgaonts are applied, there is no .question about its 
cabsonce or presence. It occurs either consciously or unconsciously, 
sys bom tic ally or unsysto::iaticall 3 ?'. Our task is to find ways to 
naico evaluation of K. tie I more C(5nscious.;an'd more, systoaatic. I 
'made sono general coanonts about o valuation’ in tho first working 

paper. Lot i.io. pick uxd ^froa, there* with a fow particular p>oints which 
I boliovc are significant for us. 

PnrpoGO of I>/a2uation , 

First, our purpose’ slioidd bo ’to tost ideas and evaluate progroies, 
not studonts. Tliis aay sco;:i like a ' fine point, but it’ is an 
iaportant distinction and aay bo difficult for tho acadoaio r.ind to 
aigost. In fornal schooling tho oaphasis is tho other vo.j round. 

V/G cxa*. line and evaJuato studonts. The instructor may wonder about 
nis course or Iiis teaching methods, particularly if his class grados 
do not fit into a boll-shapod curve, .But the, basic object remains 
to measure student achievement, and our tests ask: Ho\.' is the 

student doing? For Title I wo must ask: How is tho prograxi doing’ 

and how do our h^rpothcsvcs or ideas chock outV To what extent docs 
the program produco tho dcsirod .outcomes? To what extent arc our 
theories verified' or modified by the cxporionco of tho x>rograxi? 



B) Definitions 
* 

Second, it will bo holpful to bo clonr about the definition 
of a nunbor of torus rolatod__tp o¥P.Luati.^o^-. — 

Pur p Q 3 os arc broo.d statements of Gduc?.t:lon.?.l ends-- for c^vat^plGr 
to improve the queJLity of community life, , ,to improve the compe- 
tence of public administrators, . ,to improve local government, ', , 
to improve the quality of volunteer service. Such statoments arc 

4 

necessary starting points, but they do not give us much help in 
program dovolox)ment or evaluation. Often this was as far as many 
of the K-tlo I proposals went, ■ ^ 

Ob.loctivQs arc more explicit statement^, ,o'f traits or character- 
istics needed to achieve the jpurposos — for, example, to malto the 
public administrator understand the relationships between his office 
and the rest of • the - community. If this v;oro. for a xDrograi.i v;o would 
probably v;ant to mako this more prociso, but this should serve for 
illustrative ipurposes. It begins to define tho chojract oris tics of 
a competent administrator, a.nd incidentally, is 'based on the 
hypothesis tha.t the a.duinistrator would bo more competent if he 
viewed the community wdiolc and understood his relationship to the 
total community life. 

One tcmpta.tion in considering objectives is to include too 
many, Tliis has boon a weakness in many of Utlo I proposals. 

Tiler 0 alw'ays v/ill bo a xilotho'ra of good objectives and the xoroblcm 
is one of setting priorities ba.scd on reality, importance and 
considerations regarding axopropriato progression from one stage to 
tho noxti Proba.bly most x^^ograms would ha.vo more than a single 
objective, but it should not Iiavo so many that their achievement is 



beyond the I’calm of possibility. 



, 2 «tS 2 S 28 tUo aoliiov-od results' of an educational proeram- 
fon oxajpla, tho public ,adninistrator ecu: state the virinciiAos or 
oonocpi,o used in-u,ald.ns relationships . . ,ho can apply those pglnci- 
pics to a variety of hypothetical situations. . ,ho applies the 
prinoxples on the job. Obviously outcoaos and objactivos aust bo 
rolatod, but they aro not synonynous. In tliis oonnoction, outcoaos 

any influoaco objoctivos. iVhon wo reach the point of spool fying 

possible outcoaos, it acy bo necessary to aodify or restate objeo™ 
tivos, to bring thon in lino with roalistio outcoaos. 

C) Boh avi orol Ch n .n gn / ' • ’ . 

Eiird,. wo aust bo' clear on tho siGanlhg pjf -Wvioral ohangc.# 
At lease for our purposes, the definition "i:,'ust 'go beyond overt * 

o.otien, .boaavior aay bo active or passivoi oogl^ltivo or oonativo 

it aoy bo concornod ,kth notivatlon, . cognition, decision aaking, 
or action. Motivation is priaarily a natter of intorost or attitude, 
Cognition at an oloaontary 'lovol involves knowledge or concepts or 
principles; at a doopor lovol it is concornod v.dth an understanding 
of relationships between various phonoaona. Decision aaking involves 
aoi.,it^ to wa^so critical judgiuents and to apply knowledge and 
values. Action is aoroly tho ultiaato and aost obvious outcono of 
•an individual's oaotienal or intolloctual behavior. 

. Aay of those foras of behavioral chango is a logltlr.iato 
Objective of education. Proa' tho point of viov/o'f prograx:' planning 
or ovaluation, howovor, wo aust bo clear about ' -tho lovol of behavior 
uc o^^poct to Modify or chansc,' I havo proparod a chock list on 

r - • 

behavioral change with spocioi roforonco to inproving loc,al 
govornaont— see Appondi:: A at th.o end of tho working paper. 
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D) Tea c hill g 0:1 d Lo arnln g 

Learning is tho process by v/liich oicporicnco develops now and 
reorganizes old responses. Once wo arc clear about tho now behaviors 
we v;ant to. develop or the old behaviors wo w.ant to reorganize, wo 
are road^'' to ask: Are tho planned learning cnperionces ap'propriate 

for our purposes? 

As teachers and educators wo are ai’aro of -aost of the 
principles of teaching eaid learning, but v'o often forgot to apply 
then consciously and consistently , To ronind you of tho obvious, 
it is ir.iportant that y/c are alv/ays aware of throe inportant factors: 
Readiness to learn; opportunit.y to cbnfirn,..,’tr.in 3 f or and apiily nov; 
knowledge; and aotivation to lo.arn, 

VALthout readiness learning niLll bo indifferent at best and nay 
not occur at all. Readiness involves an appreciation of the role— 
vanco of cho learning oxporionco and 'the intellectual ability to 
deal v.'itli it« T!io ag)plication of knowledge is oqiually ir.iportant. 

The learner learns only what ho hirisolf docs; he riust be able to sec 
that v/ha/c ho has I.ca.rnod solves lorobler.is; a.nd ho must bo able to 
recognize tho siriilarity, between tho learning and the transfer 

( 

situation. Finally, there ::ust be motivation. Students learn 
only what they want t.j learn. Generally speaking there arc two 
important characteristics of m.Dtiv.ation;« Direction and intensity?-. 

Is the learning cxporienco geal-orientod and docs it have importance 
relative to altcrna.tivo acti-'/itios? A chock list on learning and 
evaluation is included as Appendi:: E, 

E) Megs uri n g tho _ 0 u t c on 0 s 

Moasuromeiit is tho final phaso of tho evaluation xorocoss — -and 
technically tho most complicated part of the task. Normally v/c 
thinlc of evaluation as based on scientifically dovolopod instruments, 
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long drawn. out and fri-htfulljr oxponsivo. Certainly it is beyond 
the .cinanci'il rocourcos^pf this pro^riMu, but at this point I doubt 
tiiac iw i/ould bo dosiroaolo ovon if xiq Iiad tho iiiionoy, Tho range 
P^^^^loLis, and inovi table variety of objectives nal^o the creation 
. of standard tests over whelming. Furthornoro, particularly at tliis 
point of exploration j v.'o must oncouro.go tlic vldost possible experi- 
mentation and discourage tho IdLnd of closure implicit in a standard- 
ized testing system. TlioroforCj I recommend that cvalua.tion talio 
the form of systematic self-study. ’• 

Solf-stud5' always presents the danger of self-decoptionv and' 
of course it c:-d.sts, bup it is not inovi ta.blp^ particilarly if tho 
study is systematic and if it is conducted by .educators whoso 
professional stance includes dotatchod soal’ch-.for truth where over 
it may lead, ' 

A systematic self-study approach should include the following 
six elements, ' • 

1 ) , Wliatovor tho device or methods used they must alv/ays- bo 

aimed at tho main question; Is the progra:'.i a.chioving the'' desired 
outcomes? _ 

2 ) The study shoild bo under talion continuously from t]ie 
boginiiing of tho program to tho end. If the ]p^^srai:i is not achiovin 

f 

its objectives, we shoild ask; Miy not? .Are the objectives 
inappropriate or unclear? Are tho oduc.a,tional experiences irrele- 
vant? Is tho organization of tho. program inadequate? Are the 
evaluation mothods invalid? 

3) Since a. self-study is largely a. subjective method, 
provision should bo made to ga.thor evidence in more than one ’vay, 
by more than one person, and from as many pooplo .as possible. 
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4) . TIio self-study i.:ust start ivith c?. cloar and precise asscss- 
i,ivjnt of til e situation in a corinuni ty and the ch.arac tori sties of the 
participants \<'itli reference to the desired outcoi:ios. This is the 
base frori .which progress, or lack of it, is aoasurod. 

5) M.iiiy uetnods .*ia.y oe used, I ar.i listing sizr fairly fainiliar 
approaches ,' but of course there are i:iany others. 

books or diaries r.iay bo kept by both staff and 
participants to provido“l)viT3nco'''o'ir progress tov.-.ard’ desired 
objectives. 

ob sorver s may be used for the ,sa.i:ie purpose. 

j playing may bo coiployod to tost changes' in' ’ 

attitudes or ability to handle or apply .knowledge. 

• . si tua'ti on s nay bo ‘created that v.lll provide a 

i.ioro valid \/a.y to tost changes in attitudes or a.bility to use nev^ 
knov/locigo or. ideas. .Role playing is always senowhat artifici.oJ., 'but 
situations can ,oe contrived so tlia.t pa,rtici pants o.rc not a.waro they 
are b-eing ^evaluated. For oxanplc, in a -pro gran designed to olininatc 
aspects of racial prejudice.^ a si'tuation v;as contrived outside the 
prograi-i .itiself in i.'hicli a. conpleto stranger expressed nrojudiced 
opinions; reactions and v;ays in which nenbors of the groun dealt 
vd.th. the situation wore observed, . . * 

v;h other they are introduced as part 
of tno learning experieneq or represent the individual's daily v;ork, 

ahking a nan*s superior how the 

indiT^dua^ IS doing. An attenpt nust bo made to discover whether 
tno desired outconos of the progran are reflected in his- norfornance. 



or cal jd xa.ni nations are s t i 1 1 
discover sene of the things ' to’ knovT, This i 



f) 

.a good w.ay to _ ^ ^ r o- x uia 

particularly uno c.o.se with cognitive skills of knowing, analysis, 
proo_oa so-ving, etc. There are luany useful ’ standard tests that 
aolp iio.^suro ohansos in attitude or tho ability, to analyze wobloas 



and arrive a c sound judsnonts. And of oour.se tests nay bo designed 

sonorally olednoa that adults arc 
ay;. there are. ways to obtain coopor.a- 

-.'I f- a::pori::iontation or in presenting tho tost as 



Pi.rt of tno learning experionco. Jlctually tho shyness is largolv 
a reticence co expose ourselves. Most of us secretively try those 

newspaper quizzes vdth the answers upside dovrn because v;o w.'^nt to 
know how \ 7 o are doing. -- 

o) Finally , it aay bo helpful to think about outcomes in terns 
of the categories of boh.avier which we nay be attonpting to change 



I 

or strcnstlion, TIio fomulatiou bolov; i.s basod on a chart dovclopcd 
by Harry L. Miller for hie book To achin :; and Lo arnin^g in Adul t 
Education , 

ACTIVE 

(Doiii") 



(Soci.al Behavior) 



CONATIVE 

(Peeliii") 



" IV. • ■ 

.(Attitudes) 



II 

('Tlio rational 
and analytic) 



COGNITIVE 

(Thinking) 






iii; . 

(Recall) 



PASSIVE 

(Knov/ing) 



Prograns in the first quadrant ala at chaii^.ng social 

bclia\aor based largoljr on beliefs or Toolings , Change is difficult 
to aoasurc. Verbal tests tend to be unroaliablo because it is one 
thing to kn.oT; the right • aiisv/or and another to r^ractico vdiat vie 
preach. Role xDlaying, contrived situations and observation outside 
tho nr ogro::: provide the best nothods. 

Prograais in the second quadrant .are concerned vdth tho dcvclop- 
nent of cognitive activity: Tho acts of judging, dotorniining 

relationships, X5j-'‘'^blcr.'i solving. In measuring thosp activities one 
nust bo aware that they arc conplox. They should hot be confused 
vdth passive cognitive ability merely to recall an analysis that was 
read or learned in tho program, Ih/idoiico of cognitive activity may 
be gathered from simiaated problems or from observation of daily 
activities. 



Programs in the third quadrant, on tho other hand, are concerned 
Ydth tho passive acquisition of knowledge, Tliis is the most familiar 
area for most of us in higher education, tho easiest to teach and 
the easiest to measure. It involves recall., recognition of facts 
or principles or tho application of jorineixiios to nov/ contexts. 
Written or oral cxa".iinations may be used to measure progress. 
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^Tlio fourth r^T^up of objcctivos roprocont basic attitudes or 
foolinsG that^a?.;/ or ::ia.y not bo ovidont in bohavior. Tliis may bo 
tiio norSt difficu3_t of all areas and tv.'o cautions should bo made. 

First many oduc-ators question \,iiotl).or clia.nging attitudes is a legi- 
timate oduca.tional objective at all. Second, seme behavioral 
scientists, argue that v;e cannot distinguish botv/o>.'n v;hat a person 
sajT’s and v;hat ho does— that v;hat he sa.ys is not a true reflection 
of attitudes, -Sven if v/o are skeptical, it maj’’ bo useful to chock 
cha.nges in attitude as peripheral, or secondary objectives. Porhaxjs 
the best device for this purpose is some form of the Lickert Scale 
v/hich ^provides the resplendent vn_th opopoortunity to a.groc or disagree 
on a five point scale — c.g, I strongly agree; I agree somov/hat; I a.m 
not concerned; I disagree sonevdiat; I strongly cU.sagrco. 

F , Summary 

In this world.ng paper v;o have considered . evaluation' as a. series 
of stages in educational programming. F?-rst, v/e must have ideas or 
hypotheses. Second, \!q must have the st;itamcnt, classification and 
selection of .objectivos and outcomes. Third, this, is followed by 
selection, planning! and organization -of appropDriato ■educational 
experiences, • Fourth, thoro ‘is evaluation, it is an on-going perocoss, 
not Something uo do a.t the end of a porogrcu-i. Our pnurposo is to 
measure change or the extent to which desired outcomes are being 

achieved, , If tnero is little or nono it may bo bocauso; ' Our hypDO— 

» > * 

theses arc faulty; the objectivos are inapp)rop^riatc , or unclear; th'o 
educational oxpDorionce is not relevant; or the evaluation instruments 
arc not valid. 

Tills is not the final sta,5e, hov/ovor; the pDroccss goes on-- the 
results of the evaluation ar.o followed by: Rcstatcmcht of hypoothosos 

and objectivos; rosoloctioii of educational cxpporioncos; and 
ro-evaluation. This continuing pDrocoss is the h'oart of the system 
piropiosod in the second v/orking pe.apior — a.nd vlthin the system it is the 
active stage agency that must pirovide direction and Integra tion 
necessary to malco the piroccss work. 
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APPENDIX A 



Ch?.iic:;o in Lb c i?.l _ G o vor nn on t 



Mo ti yg ti on ( I n t o rc3t_^_j,_^tmlc^ 

, iiunodiato lovol v;o can anticipate intorcGt in local 

interest in citizen partici iDation in local 
novor nave been hishor. At a deeper level, however, 
interest is another v.iatter. Interest r.iay bo dulled by: 
Fco_in^’s oi oeinxb’ ovcrv/holiaod by its vastnoss and roaotonoss: 
coniusion ovv:r sources of infornation; co;:riilo:-dty of political 
prob^ons; the distractions of an affluent society, 

II* Cognit ion 



the 



1 . 



Knoi:iedgo of concepts and principles is undoubtedly 



aost famliar aspect of education and evaluation. It 



s concern 
ctivc 



■to us in inprovinc lipal covornnont is not As an latiaato obio 
out as a relevant basis for other cognitive sliills, 

. ■ Ondorstanding rola'tionshins i-oquiros tlio ability to 

recognizo oouncotions anong facts or aroan^of knov;lodgo, anrlo 

tA bS r°f=roiioci;appropriato to the phcnoinona 

rrtfi roquiros botli anal5^sis and synthesis. In do-lin^'’ 

POliticai conoopS 

foi ran., uatuor) thoro arc a nui'.roor, of snoolal problonsj (a) thA 

^ibstraotj (b) people are of ton onotionally involved 

y-tosoriaod; (d) thoro is a tondonoy to classify 
-\'r'-'"to°bQ°?nn ^sis of old , e:raoricncos; (d) thoro is pi-os- 

intorosts. On tho other hand thoro 
mid L-'A f'^-ctors: (a) ]iooplo aro accustonod to classifying 

SiL that tbAv ='^t'-^prizxne IS correct; and (b) as people rocog- 

aro paa-t of i-.inny relationships, motivation to master 
abs tr'.wC 1/ concepts is neiglitoncd, 

III. ’ DECISION 

... 1 ) Critical judgTiont involves: (a) an ability to 

ability to ano'^J.yze an argunont; 

fV\ X? arrive ?.t conclusions consistent with the analysis: 

T ,!"!? 'anility to docl^vlun non-rclatod olc::ionts. Behaviors arc 

caangod oy a pro gran waich acets those four criteria. 

Values uay oo attacked by psycho tl-erawy, prowaf"‘'and''^ 
or education, Pronapada attempts to change values 11 tLout ''confr*en- 
u .tion or Uiicicrstanding of ono»s own value system. .Usin'^ different 
approaches psychotherapy and education attempt to sec Woarticulor 

in of a par'ticiaAr value' syston 

Z action in tho contont of other values. Sinco tho 

vnluo systou of participants tends to bo dooxoly ingroinod, it may bo 

it=-'lf. As in L other 

c.ro..,. It IS importanb to have the appropriate sequence in oducationol 
02:pcrionces, 'i c 
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More t!i?.n 
cosnitivo skills. 
n-?.iiipulato things 
•n locnl coniiiittco 
riot; to oiit c. n 
of to v/n f i n nn c 0 s , 



o y n Q 
t. .X (j 



niiy of the othor 
In general it involvos 
or pooplo— ^to run group 
to super vis 

’/slot tor; to use a coaputor in the inanagcncnt 



ac ti on , go os "o oy : nd 
skills or ability to 
discussions: to organize 



1 

subordinatos; to handle in incipient 
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IV 



IMPLZriENTING THE SYST'^'d: PIPE DREAM OR REALITY? 

'--y jud, 3 :'.iont , t)iis propv.>sal is njt :?.n unro:?.listic pipo droiXii* 

In tho finr.l v/orkinc papor I v/aiit to sug-ost v;liy' and how v/o laay 
boipin to iiuplonont tho syston, 

If v;o arc to bo realistic v;c oust rocognizo tho iinitations 
inposod by lack of staff and funds. This noans wo t:d^o only snail 
stoxos, one at a tino. V/ithin tho Iinitations, hov;ovcr, it is possible 
to r^nrsuc two courses, ono systoruatic a.nd tho other unsystor.iatic. 

So far v;o have boon unsystOiio.tic. Our only standards 

have been those inposod by sidnini strati ye 'rod tape. V/c have followed 
tho path of least resistance,, and bocauso. wo- are v/ithout standards, 
or syston wo _havo loarnod little that hclx^s us aalic, the nc 2 -:t decision.. 

f * ' • , * ... 

Tliis is not only unsystematic, but in th.o loiic run it is unrealistic 
to operate’ the i^roGram nlthdut a clear sense', of diroctikn. I have 
proxDOsed a system which i^rovides us with a lone ranse co-al. I believe 
it is realistic on two countis; First, it fulfills the broad x>^irposos 
of Title I of the i-aghor Education Actj and second., it is‘’possiblo 
to aT)ply it ono stop a.t a tir.io--«in other a^ords, it is orr'pediont in' 
the best sense of tho v;ord. 

It is not possible to lay out a precise x^lan specifying tho 

order in v.diich various steps shall bo tai^cn. Although tliis may be 

frustrating, it is also ono .of tho realistic aspects of the system, 

■ ■ ■ - ' * 

It seems to mo that there are a number of points whore wo might 

begin. In the paragraphs- th.at f'ell;)w I aua listing sin xoossibilitics. 

1) A sub-committoo of the Advisory Council supplomontod by 

representatives from .higher cduca,'bion could bo established to comnlote 

the stato x>la^n a,s sx^ocified by the act, Tliis rox'-ort could bo used 
as a world.ng 
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2) 111 connection v.dth the 



noict round of proposals, follow 

tho s.'^aio procedures as in the past but aake av.'drds i.lth the stipu- 
lation tnat grantees .agree to restate propos.?J.s, ' Restatonionts 

t 

snould include hypo theses, precise objectives and outcomes , and 
uo'cliods for evaluation, Prexoaration for tho res tat on out could be 
handled tnrough a series of regj.on?,l v7ork3hox:)s supplci'ientod by 
individual C'onsultation. . 

5) Organize a consortium of universities supported by a ; 

Utlo I grant to develpxo hyx^othoces v:ort!i testing. 

Organize a series of -regional woid^shops to introduce 
tho system. 

^ m * 

5) Establish a nov/slot tor as -a means ;bf communication vdth 
institutions of higher education .and community agencies involved in 
continuing education, in tho’’ beginning this need not be elaborate. 

A start could bo made v/ith an occasional momoraiidui'.i published vdien 
ti:.ve x^^^rnits and/or vdioii tlierp is something imxiortant to say, 

6) Sc-^t uj) a system for storing information so th-o.t it can be 
used, v-ZitnoUG a staff to collect and .o.nalyzc tlio data progress 

’.111 00 slo’.v, i'levcrth oloss , vlthout .any effort, much data is alrco.dy 
co.,.lng CO tho agoncy th-at boars on relevant factors such as; Sub- 
stance and method of x^Jrograms; idoas; evaluation;^ and public- and 
private agencies involved in continuing oducation. If wo knew how 
to classify .and store informati./n, -a great deal more couid be 
colloctod vlthout a groat' deal of effort. 

Any of thoso stoves aro -pos.siblo and I am sure there are many 
otnei's. As for 'bh: reality of the situation, tho ;::ajor caution is 



that v;o must bo very noclost in our oxpoctations-- progress is bound 



to bo slov/, Tho point- is: 



If wo knov; v/licro v/o arc 



hoc’.cUng and 



if what ov or 

» 

for Title i, 
sor’yicc and 



v;o do con tributes to th 
there vdLII be progress 
c o n ti liui n s o due a ti o n 



0 ostablishi'.iont of a syston 
tov:;.rd lucre effective coixuunity 
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